Recent Advancements in Augmented Reality for Robotic Applications: A Survey

Robots are expanding from industrial applications to daily life, in areas such as medical robotics, rehabilitative robotics, social robotics, and mobile/aerial robotics systems. In recent years, augmented reality (AR) has been integrated into many robotic applications, including medical, industrial, human–robot interactions, and collaboration scenarios. In this work, AR for both medical and industrial robot applications is reviewed and summarized. For medical robot applications, we investigated the integration of AR in (1) preoperative and surgical task planning; (2) image-guided robotic surgery; (3) surgical training and simulation; and (4) telesurgery. AR for industrial scenarios is reviewed in (1) human–robot interactions and collaborations; (2) path planning and task allocation; (3) training and simulation; and (4) teleoperation control/assistance. In addition, the limitations and challenges are discussed. Overall, this article serves as a valuable resource for working in the field of AR and robotic research, offering insights into the recent state of the art and prospects for improvement

Towards Exoscope Automation in Neurosurgery: A Markerless Visual-Servoing Approach

Exoscopes are a promising tool for neurosurgeons, offering improved visualisation and ergonomics compared with traditional surgical microscopes. They consist of an external scope that projects the surgical field onto a 2D or 3D monitor, providing a wider field of view and better access to the surgical site. Despite the advantages, exoscopes present some limitations, such as the need for manual or foot joystick repositioning, which can disrupt the flow of the procedure and increase the risk of user error. In this study, a markerless visual-servoing approach for autonomous exoscope control is proposed to address these limitations and enhance the ergonomics and reduce the physical and cognitive load compared with traditional joystick control. The system uses visual information from the operating field to control the exoscope, eliminating the need for markers or additional tracking devices. The proposed approach was validated using a 7-DOF robotic manipulator with a stereo camera in an eyein-hand configuration. Results showed that the system achieved 89% accuracy in detecting the target and tracking its movement with a tracking error ranging from 0.50 +/- 0.17 cm for lowspeed movements to 1.38 +/- 0.73 cm for high-speed movements. The proposed system also demonstrated improved efficiency, with a shorter execution time of 72.07 +/- 19.36 s compared with 106.52 +/- 18.50 s for the foot-joystick control. Additionally, the time out of the FoV was significantly higher in the joystick control mode and the frequency of appearance of the instrument in the centre of the image was higher when using the proposed system. The NASA TLX results indicated lower physical and cognitive load compared with the joystick control-based modality

Teleoperation Control of an Underactuated Bionic Hand: Comparison between Wearable and Vision-Tracking-Based Methods

Bionic hands have been employed in a wide range of applications, including prosthetics, robotic grasping, and human–robot interaction. However, considering the underactuated and nonlinear characteristics, as well as the mechanical structure’s backlash, achieving natural and intuitive teleoperation control of an underactuated bionic hand remains a critical issue. In this paper, the teleoperation control of an underactuated bionic hand using wearable and vision-tracking system-based methods is investigated. Firstly, the nonlinear behaviour of the bionic hand is observed and the kinematics model is formulated. Then, the wearable-glove-based and the vision-tracking-based teleoperation control frameworks are implemented, respectively. Furthermore, experiments are conducted to demonstrate the feasibility and performance of these two methods in terms of accuracy in both static and dynamic scenarios. Finally, a user study and demonstration experiments are conducted to verify the performance of these two approaches in grasp tasks. Both developed systems proved to be exploitable in both powered and precise grasp tasks using the underactuated bionic hand, with a success rate of 98.6% and 96.5%, respectively. The glove-based method turned out to be more accurate and better performing than the vision-based one, but also less comfortable, requiring greater effort by the user. By further incorporating a robot manipulator, the system can be utilised to perform grasp, delivery, or handover tasks in daily, risky, and infectious scenarios

Residual respiratory impairment after COVID-19 pneumonia

Abstract Introduction: The novel coronavirus SARS-Cov-2 can infect the respiratory tract causing a spectrum of disease varying from mild to fatal pneumonia, and known as COVID-19. Ongoing clinical research is assessing the potential for long-term respiratory sequelae in these patients. We assessed the respiratory function in a cohort of patients after recovering from SARS-Cov-2 infection, stratified according to PaO2/FiO2 (p/F) values. Method: Approximately one month after hospital discharge, 86 COVID-19 patients underwent physical examination, arterial blood gas (ABG) analysis, pulmonary function tests (PFTs), and six-minute walk test (6MWT). Patients were also asked to quantify the severity of dyspnoea and cough before, during, and after hospitalization using a visual analogic scale (VAS). Seventy-six subjects with ABG during hospitalization were stratified in three groups according to their worst p/F values: above 300 (n = 38), between 200 and 300 (n = 30) and below 200 (n = 20). Results: On PFTs, lung volumes were overall preserved yet, mean percent predicted residual volume was slightly reduced (74.8 ± 18.1%). Percent predicted diffusing capacity for carbon monoxide (DLCO) was also mildly reduced (77.2 ± 16.5%). Patients reported residual breathlessness at the time of the visit (VAS 19.8, p < 0.001). Patients with p/F below 200 during hospitalization had lower percent predicted forced vital capacity (p = 0.005), lower percent predicted total lung capacity (p = 0.012), lower DLCO (p < 0.001) and shorter 6MWT distance (p = 0.004) than patients with higher p/F. Conclusion: Approximately one month after hospital discharge, patients with COVID-19 can have residual respiratory impairment, including lower exercise tolerance. The extent of this impairment seems to correlate with the severity of respiratory failure during hospitalization